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Abstract 

We apply renormalized entropy as a complexity measure to the logistic and sine-circle maps. In 
the case of logistic map, renormalized entropy decreases (increases) until the accumulation point 
(after the accumulation point up to the most chaotic state) as a sign of increasing (decreasing) 
degree of order in all the investigated periodic windows, namely, period-2, 3, and 5, thereby proving 
the robustness of this complexity measure. This observed change in the renormalized entropy is 
adequate, since the bifurcations are exhibited before the accumulation point, after which the band- 
merging, in opposition to the bifurcations, is exhibited. In addition to the precise detection of the 
accumulation points in all these windows, it is shown that the renormalized entropy can detect the 
self-similar windows in the chaotic regime by exhibiting abrupt changes in its values. Regarding the 
sine-circle map, we observe that the renormalized entropy detects also the quasi-periodic regimes 
by showing oscillatory behavior particularly in these regimes. Moreover, the oscillatory regime 
of the renormalized entropy corresponds to a larger interval of the nonlinearity parameter of the 
sine-circle map as the value of the frequency ratio parameter reaches the critical value, at which 
the winding ratio attains the golden mean. 
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I. INTRODUCTION 



With the advent of the science of complexity, numerous complexity measures have been 
proposed. These measures can be grouped mainly in two categories: first group follows the 
route of constructing the shortest computer program corresponding to a given string. The 
well-known Kolmogorov-Chaitin 1, 2| and logical depth jsl measures fall into this category. 
The second group follows the information-theoretic approaches whose main examples can be 
given as effective complexity, thermodynamic depth jj], Shiner-Davison-Landsberg jsl and 
Lopez- Ruiz-Mancini-Calbet measures |6|. 

The information-theoretic approaches are founded on the main idea of multiplying a 
measure of order by that of disorder. In this sense, these approaches rely heavily on the def- 
initions of entropy as a measure of order/disorder. For example, Shiner-Davison-Landsberg 
[sl uses Boltmann-Gibbs-Shannon (BGS) entropy of the physical system in its definition, 
while thermodynamic depth as a complexity measure considers the entropy of the ensemble 
focusing on the entire history of the system under investigation J]. 

However, to the best of our knowledge, none of these complexity measures have been 
constructed particularly to deal with the non-equilibrium stationary states resulting from 
the external influence of a field. Such a complexity measure has been recentjiy introduced 
by Saparin et al. and applied to logistic map [3], heart rate variability 



9| and to the 

analysis of electroencephalograms of epilepsy patients [10|. This new complexity measure 
is an information-theoretic one and is called renormalized entropy, historically originating 



from Klimontovich's S-theorem (The letter S here stands for the self-organization) 
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The renormalized entropy is theoretically equivalent to negative relative entropy between a 
reference distribution and any other distribution obtained either analytically or numerically 
through a time-series analysis JigI. [i7|. However, it is not only associated with the relative 
entropy, since an additional procedure of renormalization is also introduced. The process of 
renormalization is used in order to compare two states by equating their mean energies so 
that non-equilibrium stationary states attain the same mean energy, thereby mimicking the 
ordinary closed system formalism. Combining renormalization and the relative entropy, the 
renormalized entropy decreases as the control parameter increases, indicating the relative de- 
gree of order in the system as first suggested by Haken [3] in the context of self-organization. 



On the other hand, another related issue in the context of self-organization is the exis- 



tence of bifurcations often observed in nonlinear dynamical systems: the systems possessing 
a stable fixed point become unstable as they recede away from this stable fixed point as a 
result of increasing nonlinear effects. These systems eventually pave their way to the new 
stable branches through bifurcations. For the system away from the stable fixed point, this 
process continues until the system sets in a new stationary state, thereby increasing its 
order as a signature of self-organization due to the non-linearity, dissipation and the non- 
equilibrium exhibited by the system (see the detailed discussion by Nicolis and Prigogine 



in Ref. 



19|). Physical processes such as Ravleigh-Benard [2 71. |28| . Taylor instability [29 1 



experiments, bacterial 



30| and Dictyostelium discoideum j3l| colonies fall into the afore- 



mentioned category. These systems, being open due to exchange of energy and/or matter 
with its surroundings, can not be analyzed in terms of usual H-theorem, since it is valid 
only for isolated systems. Therefore, Prigogine proposed a more general form of the second 
law i.e. AS' = AjS* -I- A^S where AiS is the entropy produced inside the system and A(.S 
is the transfer of entropy across the boundaries of the system. In this more general setting, 
AiS > whereas AgS can be positive or negative depending on the flow of energy across the 
boundaries of the system. The overall sign of AS* is determined by the interplay between 
AiS and A^S. All the aforementioned models of self-organization requires AS* < until the 
system sets in the stationary state corresponding to the most ordered pattern. 

The paper is organized as follows. In Section 2, calculation methods of the renormalized 
entropy is given. In section 3, this complexity measure is applied to discrete maps possessing 
different universality classes i.e. to the logistic map which has periodic route to chaos and 
its self similar windows like period 3 and period 5 to show its robustness and to sine-circle 
map which has periodic and quasi-periodic route to chaos. Finally, we discuss our results 
and compare the two different routes to chaos in terms of the renormalized entropy. 



II. METHOD 

A. An overview for the renormalized entropy 

Let us consider a generic dynamical map Xt+i = G{Xt,a) where t and a denote the 
number of iterations and the corresponding control parameter of the dynamical map, re- 
spectively. A small change in the value of the control parameter, Aa, yields two normalized 
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distributions i.e. f{X,a) = fo{X) and f{X, (a + Aa)) = fi{X). The corresponding Shannon 
entropies read 

SifoiX)) = - j fo{X) \nfoiX)dX , 5(/i(X)) = -J h{X) ln/i(X)rfX. (1) 

Let us now assume that the system i.e. dynamical map under scrutiny evolves in such 
a manner that the state with index 1 is evolved to through increasing order, namely, the 
system becomes self-organized as the control parameter increases. Then, setting equilibrium 
temperature = 1, the normalized Boltzmann-Gibbs distribution reads 

UX)=eM-Heff{X)] (2) 
with the following effective energy 

H,ff = -\nUX). (3) 

S-theorem by Klimontovich equates the effective energies of the concomitant states i.e. renor- 
malizes the states in order to apply H-theorem of Boltzmann to open systems, implying 
AgS* = in the second law formulation of Prigogine [26| and /o(-^) /o(^) to compensate 
for the mean energy difference i.e. turning open system into a closed one. Denoting the 
renormalized state by tilde, one can write 



/o(X) = C [UX)t^ff = C exp 



Teff J 

where C is the normalization constant. To check whether any heat intake occurs during 
self-organization, to form spatially more ordered patterns, through dissipation as a result of 
interaction with the environment, T^ff is calculated from the equality of mean energies 

J /o(X) \nfoiX)dX = j /i(X) \iifo{X)dX. (5) 
If heat intake is needed for the process of self-organization such as Rayleigh-Benard 



convection resulting in spatially ordered hexagonal patterns at the stationary state 27|, |28| , 
then one expects, comparing the equilibrium and non-equilibrium states, T^ff > 1, since 
= 1. Otherwise, we deduce that our assumption regarding the more disorderliness of the 
initial distribution is not correct, indicating the second distribution to be more disordered. 
Therefore, when this is the case, one renormalizes the second distribution. The measure 



of relative degree of order for these compared states can then be given as the difference of 
entropies 



A5 = 5(/i(X))-5(/o(X) 
which is called renormalized entropy 25 1. 



(6) 



B. Calculation of the spectrum 



We use the spectral intensities Po{w) and Pi{w) averaged over M periodograms based on 
the multiplication of the Fourier and the inverse Fourier transformation of the time series 
of Xt in the frequency domain w, instead of the density distributions fo{x) and fi{x) in 
Eqs. [HE], respectively so that 



P(w) 



'At 



At ^^a^n exp{—i2'7iwnAt) 



n=0 



(7) 



where the sequence X„ is a sufficiently long series of values that can be obtained by iterating 
a mappin g by sampling equidistant points and i is the length of the samples in every peri- 



odogram 



20|, satisfying P{w) > and Yl^i'^) ~ ^- Such a Fourier spectra eliminate the 



'zeros' in the distributions and detect different regimes of a deterministic dynamical system. 



III. APPLICATION OF THE RENORMALIZED ENTROPY 

To start with, let us consider a rf-dimensional mapping of the form 

Xi+i = G{Xt) (8) 

on some (i- dimensional phase space X. We can numerically generate data from such a 
mapping equation that describes dynamics of a specific dynamical system. We particularly 
focus on two examples. One is the logistic map given as 

Xt+i=AaXt{l-Xt) (9) 

where a G [0,4] is a control parameter and the map is confined to the interval X G [0, 1], 
which exhibits periodic route to chaos. For a = 4, the system is (semi)conjugated to a 
Bernoulli shift and strongly mixing. The other is the sine-circle map given as 
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dt+i = 9t + n-^sm {2'Kdt) mod (1) (10) 

ZTT 

where < < 1 is a point on a circle and parameter K (with if > 0) is a measure of the 
strength of the nonhnearity, whichs exhibit periodic route to chaos. It describes dynamical 
systems possessing a natural frequency uji which are driven by an external force of frequency 
UJ2 = u>i/uj2 is the bare winding number or frequency-ratio parameter) and belongs to 



2l|. Winding number 



the same universality class of the forced Rayleigh-Benard convection 
for this map is defined to be the limit of the ratio 

W = hm ^^i^ , (11) 

t-^oo t 

where {6t — Oq) is the angular distance travelled after t iterations of the map function. To 
increase the degree of irrationality of the system, one could use frequency ratio parameter 
Q corresponding to winding number W that approaches the golden mean gradually by 
following the sequence of ratios of the Fibonacci numbers (i^„/F„+i). The map is monotonic 
and invertible (non-monotonic and non-invertible) for K < 1 {K > 1) and develops a cubic 
inflexion point at ^ = for K = 1. 

Considering technical details, it is important to emphasize that we added Gaussian white 
noise into systems in Eqs. ([9]) and (fTOl) with a small intensity {Db = 10"^^), which is called 
basic noise, to every state. The low intensity of the noise is chosen so as not to influence the 
dynamics of these systems. This procedure enables a continuous spectral distribution P{w). 
After 65536 transients, we generated the discrete sequences with the length of 4096 x 18 
points separately obtained from Eq. (9) and (10). Finally, the spectrum of 18 shifted 
windows of 4096 samples (At = 0.5 s) is estimated and averaged. 



A. Renormalized entropy and its robustness at different windows for the logistic 
map: periodic route to chaos 

One of the dynamical systems possessing bifurcation properties can be cited as the lo- 
gistic map, which moves from a unique stable fixed point to the critical accumulation point 
possessing infinite periods through the periodic doubling route. Having reached the critical 
accumulation point, the system enters into the chaotic regime and from this point on it moves 
under the influence of chaotic band merging (i.e., inverse period-doubling). It should also 
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be remarked that periodic windows with different periods but albeit self-similar structures 
are also found in this chaotic regime. 

We now present the results concerning the behavior of the renormalized entropy and the 
bifurcation properties in the regions of period 2, 3 and 5. Before proceeding further, we 
note that the renormalized entropy has already been applied to the logistic map for period-2 
window by Saparin et al. in Ref. Our aim in this section is to investigate all other 
self-similar windows to check whether the renormalized entropy behaves consistently as a 
complexity measure, i.e., to check its robustness. Fig. la in particular shows the behavior 
of the renormalized entropy in the period 2 region where control parameter a lies between 
3 and 4. In this region, relative degree of order increases until the period accumulation 

(2) 

point i.e., ac ~ 3.569 as the system evolves from the equilibrium state to a new stationary 
state in accordance with the self-organization process. As a signature of the detection of 
the self-organization in this region then, the relative entropy monotonically decreases as 
expected. After the period 2 accumulation point onward until the most chaotic state with 
a = 4, the relative degree of order decreases, since band-merging (as opposed to bifurcation 
in the previous region) is exhibited by the system in this region. Therefore, the renormalized 
entropy increases in the aforementioned region in a non-monotonic manner. To sum up, for 
an open non-equilibrium dynamical system which approaches the stationary state through 
period-doubling and recedes away from the stationary state by means of band-merging, the 
behavior of the renormalized entropy conforms to the dictum of Prigogine i.e., order out 
of chaos. In Fig. lb, we zoom in the period 3 window i.e., the largest self-similar window 
in the period 2 region. The control parameter values for this window are confined in the 
interval between 3.832 and 3.856. Similarly to the period 2 window, the relative degree 
of order monotonically increases up to the period accumulation point ac ~ 3.849, and 
thereafter decreases non-monotonically. Accordingly, the relative entropy decreases up to 
the accumulation point, and begins to increase after the period accumulation point. It is 
worth noting the sudden, unexpected changes in the values of the renormalized entropy 
which signal the existence of the self-similar windows in the chaotic region. Fig. Ic shows 
the behavior of the renormalized entropy in the period 5 window, which is one of the self- 
similar windows in the logistic map. Due to the self-similarity, a behavior similar to the one 
in period 3 is exhibited by the renormalized entropy: it decreases almost up to Oc ~ 3.743, 
and then begins to increase in accordance with the decrease in the relative degree of order. 
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Finally, it is interesting to observe the turns in the relative degree of order for each of 
the three period accumulation points representing the stationary state of a non-equilibrium 
dynamical system possessing inherent fractal structure. 

B. Renormalized entropy for the sine-circle map: quasi-periodic route to chaos 

The sine-circle map can exhibit periodic, quasi-periodic or chaotic behaviors depending 
on the frequency ratio and the nonlinearity parameters i.e., Q and K, respectively. For 
< K < 1, the system dynamics is either periodic (frequency-locked) or quasi-periodic 
depending on the value of the frequency ratio parameter Q being rational or irrational. As 
the nonlinearity parameter K approaches zero, the system exhibits quasi-periodic behavior 
for all values of the frequency ratio parameter Q. 

As the nonlinearity parameter K approaches one, frequency-locked steps extend and 
occupy all Q axes where K is equal to one. In this case, there is a special fraction of Q value 
called the most irrational Qc, corresponding to the "golden mean" winding number W if 
frequency ratio parameter Q is locked to its critical value Qc- Shortly after this critical value 
on {K, Q) plane, (1, Qc) is the edge of quasi-periodic route to chaos since chaotic behavior 
can occur. All these characteristic shapes on {K, Q) plane is called "Arnold Tongues" in the 
literature. For the K > 1 region where the nonlinearity parameter K is dominant on the 
system dynamics, there could be periodic regions with different periods, chaotic regions, and 
so edges of periodic route to chaos. Also, for this region, there could be periodic windows 
possessing same universality class with the logistic map. 

Fig. 2 shows the behavior of the renormalized entropy and the bifurcations of the sine- 
circle map for ^4 = 0.592523777531..., l^s = 0.611736528912... and Qc = 0.606661063469... 
obtained from Eqs. fllOtllip . respectively where the nonlinearity parameter K lies between 
zero and 2.5. The reference state for the renormalized entropy is chosen to be the one with 
K = and = where the system evolves towards a unique stable point. Note that the 
degree of irrationality of the system increases as one moves from Fig. 2a towards Fig. 2c. 

In each of the aforementioned figures, the oscillatory behavior of the renormalized entropy 
is observed when the system is in the quasi-periodic regime. This oscillatory behavior is 
exhibited when K G [0,0.80], [0,0.95], [0, 1] for the cases ^2=^4, ^5 and Q^, respectively. It 
is interesting to note that the interval of the nonlinearity parameter K increases in regard 
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FIG. 1. The bifurcation diagram (upper panels) and renormalized entropy (lower panels) for 
period 2 (a), period 3 (b) and period 5 (c) of the logistic map. 
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to the oscillatory behavior of the renormalized entropy as the value of the frequency ratio 
parameter reaches the critical value Qc, at which the winding ratio W attains the golden 
mean. As a result, the renormalized entropy can detect the quasi-periodic regime as can 
be seen from Fig. 2. It is worth noting that the Lyapunov exponent is zero for all quasi- 
periodic regions as well as periodic regimes at the bifurcation points |23|. In this sense, the 
renormalized entropy is superior to the Lyapunov exponent, since the renormalized entropy 
behaves in a distinct manner in both of the aforementioned regions. Many chaotic and 
periodic regions with different periods are present in Fig. 2 for the nonlinearity parameter 
K values K G [0.80, 2.50], [0.95, 250], [1, 2.5] corresponding to ^2=^24, ^5 and Qc, respectively. 
The renormalized entropy always attains values close to zero in these intervals for the chaotic 
regions, while it decreases with the increasing number of periods in the periodic regions until 
it reaches the edge of chaos. This can be considered as the signature of the relative degree 
of order within the system. 

It is well-known that the sine-circle map is in the same universality class as the logistic 
map for K G [3.19,3.81] when Q = Qc- Fig- 3 shows the bifurcation and the renormalized 
entropy for this particular window. The renormalized entropy behaves exactly as it does in 
Fig. la for the logistic map, thereby indicating that different dynamical maps exhibit same 
behavior in the regions falling into the same universality class. 



IV. CONCLUSIONS 



Despite the presence of many different complexity measures, the ones enabling a local 



comparison of the distributions are quite few (for a recent example, see Ref. 2^)- One 



such measure of relative nature is the renormalized entropy introduced by Klimontovich, 
Kurths and coworkers 7-1^. In this work, the renormalized entropy is used to analyze 
the logistic and sine-circle maps. In the former example of the logistic map, renormalized 
entropy decreases (increases) up to the accumulation point (after the accumulation point 
until the most chaotic state) as a sign of increasing (decreasing) relative degree of order 
in all the self-similar periodic windows, thereby proving the robustness of this complexity 
measure. By robustness, we emphasize the similarity of the behavior of the renormalized 
entropy in all the self-similar windows, therefore removing the doubt concerning a possible 
accidental feature of the renormalized entropy as a complexity measure. On the other hand, 
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FIG. 2. The bifurcation diagram (upper panels) and renormalized entropy (lower panels) for 
(a), (b) and f^c (c) of the sine-circle map. 
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FIG. 3. The bifurcation diagram (upper panel) and renormalized entropy (lower panel) of the 
region possessing the same universality class with the logistic map for of the the sine-circle 
map. 



The aforementioned observed changes in the renormalized entropy are reasonable, since the 
bifurcations occur before the accumulation point, after which the band- merging, in opposi- 
tion to the bifurcations, is exhibited. On top of the precise detection of the accumulation 
points in all these windows, we see that the renormalized entropy can detect the self-similar 
windows in the chaotic regime by exhibiting sudden changes in its values. For the sine-circle 
map, on the other hand, the renormalized entropy detects also the quasi-periodic regimes 
by signaling oscillatory behavior particularly in these regimes. Moreover, the oscillatory 
regime of the renormalized entropy corresponds to a larger interval of the nonlinearity pa- 
rameter of the sine-circle map as the value of the frequency ratio parameter reaches the 
critical value, at which the winding ratio attains the golden mean. Lastly, we remark that 
the renormalized entropy is superior to the Lyapunov exponent as a complexity measure, 
since the renormalized entropy can detect the quasi-periodic regimes as well as the periodic 



regimes at the bifurcation points in a distinct manner whereas the 



zero for both of these regions, hence detecting no difference at all 
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23| 



jyapunov exponent is 
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